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DETAILED ACTION 
Response to Amendment 

1 . In response to the notice of a non-compliant amendment from 4/1/2009 and the 
Final Office Action from 7/9/2008, the applicant has submitted an amendment, filed 
5/5/2009, adding significant portions to the specification and making no amendment to 
the claims, while arguing to traverse the art rejection for numerous reasons 
(Amendment, Pages 55-68). Applicant's arguments have been fully considered, 
however the previous rejection is maintained due to the reasons listed below in the 
response to arguments. 

2. In response to the amended specification beginning at Page 49, Line 1 (where 
the applicant has corrected element numbers that did not correspond to the correct 
drawing elements shown in Fig. 4A, Amendment, Page 53), the examiner has 
withdrawn the previous corresponding objection to the disclosure directed towards 
minor informalities (OA from 7/9/2008, Page 9, item 9). 

3. In response to the specification amendment beginning at Page 58, Line 5 (where 
the applicant has specified the specific figure number, Amendment, Page 54), the 
examiner has withdrawn the previous corresponding objection to the disclosure directed 
towards minor informalities (OA from 7/9/2008, Page 9, item 9). 
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4. In response to the specification amendment beginning at Page 65, Line 22 
(where the applicant has corrected the element numbers assigned to the analyzer filters 
and analyzer, Amendment, Page 54), the examiner has withdrawn the previous 
corresponding objection to the disclosure directed towards minor informalities (OA from 
7/9/2008, Page 9, item 9). 

5. In response to the specification amendment beginning at Page 70, Line 4 (where 
the applicant has corrected the element number assigned to the command input 
resolver), the examiner has withdrawn the previous corresponding objection to the 
disclosure directed towards minor informalities (OA from 7/9/2008, Page 9, item 9). 

6. In the examiner status section, the applicant first argues that in the Office Action 
from 7/9/2008, the examiner asserts that the applicant's traverse regarding the previous 
rejection from 11/21/2003 is moot because the amended claims necessitated new 
grounds of rejection and alleges that in making this statement, the examiner has 
impliedly admitted that the applicant did not make amendments to overcome the prior 
art utilized in the rejection from 1 1/21/2003 because this cited art did not render the 
claims unpatentable (Amendment, Page 26). 

In response, the examiner points out that no such admission was made impliedly 
or otherwise. In the applicant's response filed on 4/3/2008, the applicant submitted 
significant amendments to independent claims 1 , 3, and 4 and added new claims 5-44. 
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Since the claims now included subject matter which changed their scope and was not 
previously recited, these amendments necessitated further search/consideration and a 
new grounds of rejection. This matter is to which the aforementioned examiner's 
statement is referring and makes no type of implied admission as the applicant has 
alleged. Thus, a new grounds of rejection was necessitated because the claim scope 
had significantly changed not because the previous prior art from the 1 1/21/2003 Office 
Action failed to render the applicant's claimed invention from 7/13/2001 unpatentable. 

7. Next, the applicant continues to allege that the applicant disagrees that a 
continuation was discussed (Amendment, Page 26). In response, the examiner notes 
that the previous position set forth in the Final Office Action from 7/9/2008 (Pages 2-3) 
is maintained. The applicant also makes allegations regarding internal documents 
related to USPTO policy (Amendment, Page 26). The examiner is unaware as to which 
documents the applicant is referring. Examining procedures are detailed in MPEP 700. 
Otherwise, these allegations fail to overcome the objections/rejections of Office Action, 
and thus, are considered moot. 

8. In regards to the Election/Restriction section of the Final Office Action from 
7/9/2008 (Page 3), the applicant makes the argument that only claim 1 1 was withdrawn 
while claim 37 was not withdrawn or rejected and should be allowable (Amendment, 
Page 27). In response, the examiner clarifies that claim 37 was listed with claim 1 1 
under a section heading entitled "Election/Restrictions" (Amendment, Page 3). Further, 
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discussion was made as to why claim 37 was directed to an invention different from that 
originally claimed ("directed to multi-system processing", Page 3). Thus, although claim 
37 was omitted in the "withdrawn from consideration" statement as a result of a 
typographical error, it was clear that it was intended to be included in this statement 
because it was bolded under the "Election/Restrictions" section of the Office Action and 
described as to how it differed from the originally claimed invention. Therefore, this 
claim is clearly not allowable, but has been withdrawn from consideration as being 
directed to a non-elected invention. In response to the applicant's 
comments/arguments directed towards their being prevented from claiming certain 
aspects of their invention (Amendment, Pages 27-28), the examiner notes that the 
claimed invention is directed towards conversant speech recognition based 
embodiments. As non-speech commands require a different type of processing than 
speech and receive a different classification in the art (i.e., gesture, etc or image 
recognition, class 382), the restriction of claims 1 1 and 37 is proper. Also, it is noted 
that the applicant's discussion of claims 1 1 and 37 is moot, as it has been canceled in 
the presently filed amendment. 

9. The applicant has addressed the 35 U.S.C. 112, first paragraph rejection directed 
to claim 28 or the 35 U.S.C. 112, second paragraph rejection directed towards Claims 
25-26. As such, these rejections have been maintained. 
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10. In response to the objection directed towards claim 1 and 3-4, which recite "use- 
based" objectives, the applicants argue that "the terms use and used based are 
disclosed throughout the originally filed specification" and believes that the examiner 
has already entered amendments including this term (Amendment, Page 28). 

In response, the examiner has carefully reviewed the initially filed specification 
from 7/13/2001 and was able to find only one instance of "use-objective". Only page 35 
of the originally filed specification even mentions use-objective and seems to contradict 
the "user-objective" terminology which populates the entire specification. Thus, it is 
believed that this instance of "use-objective" is actually a typographical error and has 
been objected to below. As per the entered amendment, the examiner notes that the 
only amended portion of the specification that includes this terminology is on Page 17 of 
the amendment from 4/3/2008 and is merely an amendment of the aforementioned 
original Page 35. Thus, the examiner did not enter this claim language as being 
permissible and merely overlooked an apparent typographical as part of a lengthy 
disclosure. As such, this previous objection directed towards claims 1 and 3-4 has been 
maintained. It is recommended that the applicant amend "use-based objective" to - 
user-based objective-- in order to clarify their claim language as being consistent with 
the originally filed specification and overcome this objection. 

11. In response to the applicant's amendment of claims 1 6, 27-31 , the examiner has 
withdrawn the associated previous objections directed towards minor informalities. 
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12. The applicant next turns to the objections to the specification directed towards 
new matter issues. Before addressing the specific objections, the applicant first argues 
that that no new matter was submitted in the previous amendment filed on 4/3/2008 and 
the examiner should have known that there was no new matter filed in the case due to 
the alleged support provided by the applicant in the corresponding response 
(Amendment, Page 29). The examiner respectfully disagrees. Consideration of the 
applicant's support for the amendment and the originally filed specification was made 
and the substantial additions to the original specification were still found to contain new 
matter for the reasons clearly provided/described on Pages 4-7 of the previous Office 
Action. These new matter issues result from significant additions/expansions to the 
originally filed specification that go beyond a simple clerical/grammatical touch up of the 
original specification, and thus, includes new matter. The examiner will address the 
applicant's specific arguments pertaining to each new matter issue below. 

1 3. With respect to the new matter objection of the specification directed towards 
page 58 starting at line 5, the applicant argues that the examiner's objection is 
erroneous because the only example provided is the entertainment systems and that 
the repeating of Fig. 13b is superfluous (Amendment, Page 30). In response the 
examiner notes that the entertainment system-related items are all directed towards 
new matter. The secondary mention of 13b is merely a typographical. Figs. 3a and 13b 
are nearly identical and the repeat of Fig. 13b should actually be Fig. 3a. This error has 
been corrected in the below objection. Going back to the originally filed drawings that 
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were supported in the originally filed specification, it can be seen that Fig. 3 is most 
closely related to the aforementioned figures. Comparing the original Fig. 3 to the 
aforementioned figures, it can be readily seen that the applicant has added a plurality of 
elements that were not shown in this figure or even contemplated in the originally filed 
specification (i.e., entertainment systems shown on the right side of the drawing as well 
devices associated with entertainment systems such as the remote controller shown on 
the right side of Fig. 3a and 13b). As further evidence that these entertainment devices 
are directed towards new matter, the examiner points out that the applicant added new 
matter in describing Fig. 3 in the form of a new sentence added to the specification 
beginning at page 30, line 1 1 : "Lesser related intermittent tasks may also include, for 
example, temporarily interrupting using computer 312 of FIG. 3a to control light 31 3d, a 
television, stereo/gaming equipment, and so on." Fig. 13b also includes the elements of 
Fig. 3a, which as described above, are directed to new matter. So when the 
amendment starting at page 58, line 5 references these figures, they include new 
matter. Thus, this previous objection is proper and maintained. 

The applicant has also pointed out where the brief description of these figures 
and other similar figures directed towards new matter was not objected to (Amendment, 
Page 30). With this omission brought to the examiner's attention, these additions to the 
specification have also been objected to below for being directed towards new matter. 
The applicant presented significant amendments on 4/3/2008 requiring significant 
review and the alleged omitting of these brief descriptions from the previous objections 
was not an admission of any type by the examiner (alleged by the applicant, 
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Amendment, Page 31) as is clear from the fact that the detailed descriptions of these 
drawings were explicitly objected to as being directed towards new matter (Prior Office 
Action, Pages 4-7). Clearly, if the detailed descriptions were directed towards new 
matter their associated brief descriptions would also be directed towards new matter. 
Once again, the examiner does find these brief descriptions to be directed towards new 
matter and they have been included in the below objections. 

Next, the applicant presents arguments as to why amended Fig. 13b is not 
directed towards new matter. First, the applicant argues that the specification portion in 
question on Page 58 does not use the language "entertainment systems" (Amendment, 
Page 31). The examiner notes that this language was merely provided in order to 
specify a grouping of the added types of entertainment-related items shown in Fig. 13b. 
The applicant next argues that the original specification does recite an "entertainment 
system" (Amendment, Page 31). The examiner does find this example of a generic 
entertainment system. The problem with Fig. 13b as well as 3a is that the applicant has 
added specific types of entertainment systems beyond this generic citation (i.e., game, 
stereo systems, etc.). In the amended description of Fig. 3a, which starts at Page 30, 
Line 1 1 , the applicant added these specific types. Since 1 3b and 3a include these new 
elements not described in the original specification, these drawings as well as their 
citations in the specification are directed towards new matter. For further comparison, 
see the original Fig. 3a and observe the significant changes that were made in the 
previous amendment. Third, the applicant argues that Fig. 13b is supported by the 
original Fig. 3a (Amendment, Page 31). This argument is not convincing because there 
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have been significant additions to this figure in the amendment filed on 4/3/2008, as is 
described above. It is worth noting that the applicant even appears to admit that some 
of these elements are clearly new ("game or gaming machine", Amendment, Page 31). 
In response to the applicant's issue of "entertainment system" being supported by the 
original specification (Amendment, Page 31), the examiner notes that, as is described 
above, it is the specific added types of entertainment devices that is the issue 
necessitating the new matter objection. Thus, these arguments have been fully 
considered, but are not convincing and Fig. 13b, 3a, and the citations/mentions thereof 
are directed towards new matter. 

Next, the applicant presents arguments directed towards the second instance of 
13b (Amendment, Page 32). As was pointed out above, as a result of a typographical 
error, one of these instances should be listed as Fig. 3a. 

In addressing the specification objection directed toward Fig. 15f, the applicant 
argues that the amendment starting at Page 58, line 5 does not reference this figure 
(Amendment, Page 32). In response, the examiner notes that in fact, this objection 
should be considered in addition to the objection directed towards the amendment 
starting at Page 58, Line 5. In the amendment from 4/3/2008, starting at Page 8, Line 3, 
Figs. 15f and g are described with respect to their new matter descriptions as is detailed 
in the previous Office Action (Pages 4-5). For Fig. 15f, the description includes 
biometric, movement recognition for user identification (which relates to image 
recognition), which was not described in the originally filed specification. Thus, this 
objection has been maintained, but separated for clarity below. In support of Fig. 15 f 
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and its associated descriptions as being supported by the original specification, 
however, the applicant argues that on original page 67, line 21- page 68, line 29, Fig. 15 
f is supported (Amendment, Page 33). While the examiner does/did see support for 
some of the user identification types described with respect to 15f, the issue remains 
that, similar to Figs. 13b and 3a above, the applicant appears to have expanded, 
without support, what was originally filed. In this instance, there is no support in the 
original specification for proximity-based identification as the original specification is 
silent as to this type of user identification. Thus, claim 15f and its associated description 
are directed towards new matter is maintained. 

In response to the applicant's comments regarding the alleged omission of the 
brief description from the specification objection (Amendment, Page 33), the examiner 
notes that, as was pointed out above, this omission has been clarified, is not an 
admission, and that the objection of the Page 58, line 5 amendment includes the 
objection of the brief description for the same reasons that were pointed out above with 
regard to Fig. 15f. For support for Fig. 15g, the applicants argue that at Page 27, Lines 
8-13 of the original specification, it was noted that common user mis-recitations can be 
trapped (Amendment, Page 33). In response, the examiner notes that this portion of 
the specification only makes a mention of this function in a general fashion. On the 
other hand, Fig. 15g shows/describes specific elements that have been 
expanded/added from the original passage. The specific elements associated with this 
function were not described in the originally filed specification (such as a specific 
trapping interpreter and memory). The memory mentioned by the applicant on Page 34 
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of the amendment stores all commands and is not trapped command specific as shown 
in Fig. 15G. Also, the multi-output/result of the trapping interpreter is not described. 
Thus, Fig. 15g and its associated references are directed towards new matter. 

The applicant next argues the specification objection directed towards the 
amendment beginning at Page 22, line 14. In response to the allegation that the 
examiner admitted that the original Fig. 3a included gesture recognition, the examiner 
notes that the only admission that was made was that a user is shown next to another 
user pointing to a computer and nothing beyond (Prior OA, Page 5). There was not a 
device for making such a gesture or any of the other recognition types in the original 
specification as was added in the amendment from 4/3/2008. More specifically, original 
Fig. 3a only showed one user pointing to a computer system while another user sits 
nearby. In the amendment to the specification starting at Page 22, line 14, the applicant 
added a significant description that could not be gleaned from the aforementioned 
simple illustration. In the original description of Fig. 3a, all that was described was a 
user instructing another user or controlling a machine. There was nothing discussed 
regarding the specific gazes/gestures used to control a machine that was recited as the 
applicant argues. 

Next, the applicant argues that the original Fig. 3a is more extensive than the 
examiner admits. More specifically, it is argued that the figure shows various items for 
control as well as a user head and body particularly positioned. The applicant also 
refers to the original specification Page 115, lines 24-31 for support (Amendment, 
Pages 35-36). In response, the examiner once again notes that the applicant appears 
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to be adding meaning that was not in any way supported by the original specification. It 
is uncertain how looking at a drawing of two users surrounding a system the idea of 
various recognition types could be supported. It is unclear how the users body/head 
positions relate in any way to any type of command. Also, the cited portion of the 
specification merely notes that modifications may be made to the applicant's invention 
and does not describe any of these specific recognition types. Thus, this argument has 
been fully considered, but is not convincing. The applicants arguments/comments 
directed towards the interfaces in the specification are not directed to the 
aforementioned informalities and thus do not overcome this objection (Amendment, 
Page 36). 

The applicant's arguments that the specification can be amended to include 
recitations pertaining to the original drawings (Amendment, Pages 36-38) are not 
convincing. For the above noted reasons, the applicant has expanded general 
recitations/illustrations with very specific examples that were not supported by the 
specification. The applicant's amendments are not merely simple descriptions of what 
was already shown in a general drawing or recitation in the originally filed specification 
and are directed towards new matter. The applicant next provides specific examples as 
to why the amendments starting at Page 22, line 14 are supported in the originally filed 
specification (Amendment, Pages 38-39). In response, the examiner notes that while 
there does seem to be support for the added pointing device, the specification is still 
completely silent as to any type of head, body, or gaze recognition. The below 
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objection has been altered to this effect, but the applicant's arguments are ultimately 
unconvincing. 

Next, the applicant traverses the new matter objection to the specification 
amendment beginning at Page 27, Line 4. Specifically, the applicant relies on the 
above arguments directed towards Fig. 15g (Amendment, Pages 39-40). In regards to 
such arguments, see the above response directed towards Fig. 15g. 

Next, the applicant traverses the new matter objection to the specification 
amendment beginning at Page 30, Line 1 1 . Specifically, the applicant first argues that it 
is not clear whether the examiner admits or objects to the general or particular 
examples (Amendment, Page 40). The examiner notes that it is the specific types of 
devices controlled under the terminology lesser related intermittent tasks as is clear 
from the objection at the bottom of Page 5 of the previous Office Action. Again, the 
applicant has added specific examples that were not supported in the originally filed 
specification. The original specification did not at all mention controlling lights, 
television, stereo/gaming systems etc as is set forth in the aforementioned amended 
passage. 

The applicant continues to argue the support of the amendment starting at Page 
30, Line 1 1 by noting that the surrounding disclosure should be considered as support 
and provides these citations. The applicant argues that the added text thus only 
exemplifies the original definitions and examples (Amendment, Pages 40-41). In 
response, the examiner notes that the original specification provides a general 
description of lesser related intermittent tasks. The original specification does not 
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describe or support the specific systems that the applicant has added in the previous 
amendment. While Page 28, Lines 17-20 would seem to support some type of stereo 
interfacing, it seems that it is in regards to guidance rather than control. Nowhere in the 
original specification is a gaming system discussed in any fashion and stereo or light 
control is simply not recited. Although the applicant's specification regards general 
machine control, it does not support the specific examples added by the applicant in the 
amendment from 4/3/2008. These additions are not simply obvious variants of the 
general descriptions or inherent because not every computer system would include the 
very specific components which the applicant has added in their amendment. Thus, 
since there is no support for these specific examples or disclosure of the devices 
involved, the new matter objection is proper. 

The applicant next argues that each of the specific machines is supported by 
original Fig. 3a (for example, a game machine) and its description which generally 
describes an entertainment system (Amendment, Pages 41-42). In response, the 
examiner has once again review Fig. 3a. Fig. 3a is a simple illustration that shows a 
user pointing to a computer screen while another user sits nearby. It also depicts what 
appears to be an office setting featuring a desk. Nowhere in this picture is anything 
even resembling a gaming system present. The specification makes no mention of a 
game or any type of gaming. Thus, gaming systems are not supported by this original 
figure. In regards to the specification, the examiner notes that the applicant has taken a 
very broad term and expanded it to encompass a very specific definition including very 
specific items. These items were not supported in the original specification, and thus, 
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are directed towards new matter. Thus, this argument has been fully considered, but is 
not convincing. 

Finally, in regards to the amendment starting at Page 30, Line 1 1 , the applicant 
argues that controlling of the additional components is clearly set forth in the original 
disclosure because the specification provides examples of device control and continues 
to provide these examples (Amendment, Page 42). In response, the examiner notes 
that while the specification does detail general device control and even provides some 
examples in the form of a computer, the originally filed specification in no way 
encompasses the additional components added by the applicant in the amendment 
from 4/3/2008. The applicant has attempted to expand/change the scope/definition of 
what is encompassed by these general descriptions in a manner that is not supported 
by the originally filed specification by adding very specific components that were not 
described. Thus, this alleged support has been fully considered, but is not convincing in 
withdrawing the corresponding new matter objection. In response to the applicant's 
arguments that this objection is confusing and may not have been intended by the 
examiner (Amendment, Page 43), the examiner notes that such an objection was 
clearly intended and clearly pointed out which is why it was included under the section 
heading directed towards specification objections. If the objection was unintended it 
would not have been made. 

Next, with regards to the objection of the amendment to the specification starting 
at Page 32, Line 29, the applicant argues that Fig. 13B, which is referenced in this 
passage is taken from originally filed Fig. 3A and is not directed to new matter under 
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similar rationale (Amendment, Page 43). The applicant also argues that gaming 
systems were not considered in this objection (Amendment, Page 43). In response, 
please see the appropriate responses above, which explain why Figs. 13B and 3A are 
directed towards new matter. Also, it is pointed out that on Page 6 of the Office Action 
from 7/9/2008, "gaming systems" was explicitly mentioned in this objection as well as in 
regards to other instances of these figures in the specification. Thus, these arguments 
have been fully considered, but are not convincing. 

The applicant argues that the amendment to the specification beginning at Page 
47, Line 10 is not directed towards new matter because that this portion of the 
specification has been amended to reinstate the deleted portion, which necessitated in 
the objection (Amendment, Page 44). If this specification amendment beginning at 
Page 47, Line 10 pertaining to the context and task engine definitions was 
corrected/restored accordingly, the examiner would have withdrawn this new matter 
objection to the specification. As this amendment has not been made in the present 
response, however, this objection is maintained. 

The applicant argues that the amendment to the specification beginning at Page 
47, Line 10 is not directed towards new matter because it is alleged that Fig. 10C-D 
were respectively supported by originally filed Figure 5 that was labeled "VI Converter" 
and a non-numbered drawing labeled "New CMD Conversion" (Amendment, Pages 44- 
45). The applicant also argues that these drawings having been omitted in an objection 
is an admission by the examiner that the drawings and the descriptions thereof are not 
directed to new matter (Amendment, Page 45). In response to the applicant's 
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comments directed to the brief description, see the corresponding response above, 
which addresses these allegations multiple times. Moving to the applicant's alleged 
support for these drawings then, it appears that Fig. 5 of the originally filed specification 
does directly correspond to Fig. 10C, with Fig. 10C being a machine rendered version of 
the original hand sketch. Likewise, the previously unnumbered Fig. labeled "New CMD 
Conversion" directly corresponds to the previous Fig. 10D with the exception of the 
"Interface Glue" which was not shown in the original figure "New CMD Conversion" or 
even mentioned in the original specification. In the present amendment, the applicant 
has amended Fig. 10D to change element 1033b back to its original name in the form of 
"Iteration Check". Based on these amendments and the aforementioned original 
figures, the examiner notes that Figs. 10C-D are not directed towards new matter and 
their reference in the amendment to the specification beginning at Page 47, Line 10 is 
not directed towards new matter. Accordingly, this objection has been withdrawn. 

The applicant argues the new matter objection to the specification amendment 
beginning at Page 58, Line 5 by alleging that the recitation of Fig. 13b is not directed 
towards new matter for reasons similar to the amendment to specification beginning at 
Page 30, Line 1 1 and also note that similar Fig. 3a has been omitted from this objection 
(Amendment, Pages 45-46). In response to such arguments, see the above response 
directed towards the amendment to specification beginning at Page 30, Line 1 1 . Also, 
in regards to the omission of Fig. 3a in this objection, the examiner notes that it was 
clear that Fig. 3a was considered to contain new matter as per its multiple inclusions in 
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the new matter objections. Its exclusion was a result of a typographical oversight, and 
has been corrected in the below objection. 

The applicant next turns their attention to the objection directed towards the 
amendment to the specification beginning at Page 62, Line 17. In this objection, there 
was an issue taken with the amendment because very specific definitions were added 
to the specification that described the parser and simulator/emulator/other machine. 
The applicant first argues that the examiner admitted that the originally filed Fig. 6a 
included the parser and the connected simulator/emulator/other machine (Amendment, 
page 46). The examiner has again reviewed original Fig. 6a and agrees that such 
elements are present (parser- original element 526 and simulator/emulator/other 
machine- original element 531). The applicant than traverses that the added 
descriptions associated with these elements are not directed to new matter. In the case 
of the parser, the applicant notes that the added definition of the parser is in line with 
the commonly accepted/utilized definition of a parser, which is a means for breaking an 
input into defined portions (Amendment, Page 47). The examiner has considered these 
arguments and since the scope of the applicant's parser definition does not at all go 
beyond the accepted dictionary meaning of this word and since the parser was shown in 
original Fig. 6a, the parser definition added to the specification is not directed towards 
new matter. In the case of the simulator/emulator/other machine, the applicant argues 
that original Fig. 6a clearly shows a connection between the executor components and 
the simulator (Amendment, Pages 47-48). In response, the examiner finds that original 
Fig. 6a does show this connection and since the added definition of this component 
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does not go beyond describing the shown connection, the examiner notes that this 
amendment is not directed towards new matter. Accordingly, the new matter objection 
directed to the amendment of the specification beginning at Page 62, Line 17 has been 
withdrawn. 

Lastly, in regards to the new matter objections directed towards the amended 
specification, the applicant turns their attention to the amendment beginning at Page 65, 
Line 22. The applicant begins their arguments by first arguing that the characteristics 
described in this amendment are inherent/supported in the originally filed disclosure and 
proceeds to argue the merits of the command input resolver. For support of the 
command input resolver, the applicant argues that the device is called a command input 
resolver and thus would inherently resolve input commands based on the original figure 
76 (Amendment, Page 49). The examiner agrees that the original term "command input 
resolver" does inherently support resolving an inputting command. The issue with this 
amendment is that it includes a very specific manner in which the resolver operates on 
the input (i.e., according to "root base, designation, extension" etc.). The sections of the 
specification relied upon by the applicant (Amendment, Pages 49-51) only describe 
general command/dictation or partial command/dictation resolution processes. 
Nowhere in these passages is there a connection between this component relying on 
the aforementioned considerations to resolve a user input. It is this definition that was 
extended from the applicant's originally filed specification that was not supported and is 
the cause of the new matter issue with respect to the command input resolver. Thus, 
this argument has been fully considered, but is not convincing. For support of the 
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"machine switcher 744" the applicant argues that the machine switcher inherently 
"provides for machine switching" and points to various portions of the specification 
which allegedly teach its corresponding functions (Amendment, Page 51). In response, 
the examiner has reviewed these portions of the originally filed specification and finds 
them at most to be only general commentary relating to some of these functions. These 
functions are never linked, in the originally filed specification, to a single component in 
the form of a "machine switcher". It is unclear that operations such as cueing, muting, 
etc. would even be relating to machine switching. While the examiner would note that a 
machine switcher would inherently perform machine switching, the plurality of other 
operations that the applicant alleges were supported as being associated with the 
machine switcher in the originally filed specification are non-existent. Thus, the addition 
to the specification directed to the machine switcher contains new matter. For support 
of the Data carrier filler, the applicant argues that this component "clearly inherently 
provides for conducting data carrying/filling" (Amendment, Page 51). As the applicant's 
added definition to this component does not expand upon the inherent statement, which 
the examiner notes would be supported, this particular component is not directed to new 
matter as the original Fig. 7e contained a data carrier/filler (Element 646). The below 
corresponding objection has been modified to this effect. For support of the 
"choice/error feedback engine" the applicant points to various portions of the 
specification which allegedly teach its corresponding functions (Amendment, Page 51). 
The cited portions generally discuss providing a user with information and with expected 
responses. They in no way connect determining, generating, or directing, user/machine 
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feedback to a specific component known as the "choice/error feedback engine", they 
are merely general statements to which the applicant has added an expanded definition 
through the specification amendment. Thus, this argument has been fully considered, 
but is not convincing. Finally, for support of the "machine controls" the applicant argues 
that this component "inherently provides for determining or conducting machine control" 
(Amendment, Page 51). As the applicant's added definition to this component does not 
expand upon the inherent statement, which the examiner notes would be supported, 
this particular component is not directed to new matter as the original Fig. 7e contained 
a data carrier/filler (Element 648). The below corresponding objection has been 
modified to this effect. 

Accordingly, the below new matter objections to the specification have been 
maintained for at least the preceding reasons. 

14. The applicant next turns their attention to the objections directed towards the 
drawings. The applicant first notes that they have filed several new portions of the 
specification in order to overcome objections directed to missing descriptions 
corresponding to drawings (Amendment, Page 52) (see also Prior OA, Page 7, Item 7). 
While these previous objections have been overcome, it is noted that some of these 
additions have raised the issue of new matter. Objections to this effect have been set 
forth below. It is also of note that the applicant has provided no indication that the 
amendments do no contain new matter. The applicant next argues that Fig. 17 has 
been amended so that its element numbers differ from those in Fig. 16 (Amendment, 
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page 52). In response, the examiner has withdrawn the previous corresponding 
objection to the drawings. 

The applicant next turns to the objections of Figs. 3A, 1 0C, 1 0D, 1 3B, and 1 5G. 
For Fig. 3A (which was intended by the previously noted "Fig. 3" and has been clarified 
below), the applicant argues that the objection was unclear and lacks sufficient detail 
and that changes to the original figure were minor and non-substantiative (Amendment, 
Page 52). The examiner respectfully disagrees. It is apparent in comparing the original 
Fig. 3A to the current Fig. 3A that substantial changes have been made and objects that 
were not even shown in the original figure. These changes are, thus, much more than 
non-substantial as the applicant alleges. In the objection, these additions were clearly 
indicated as "different items no originally shown" (Prior OA, Page 8, item 8). Therefore, 
this argument has been fully considered, but is not convincing. 

The applicant next traverses the objection of Fig. 10C by arguing that the 
applicant is unable to determine the reason the objection is raised (Amendment, Page 
52). In response, the examiner notes that since Fig. 10C corresponds to original Fig. 
"New CMD Conversion" (which was previously unnumbered), this objection has been 
withdrawn. 

The applicant has not addressed the new matter objection directed to Fig. 10D in 
this section, which included the element "Interface Glue", however this objection has 
been withdrawn because in the present drawing amendment the applicant has changed 
this element back to "Iteration Check" which is the terminology utilized in the original 
figure. 



Application/Control Number: 09/905,590 Page 24 

Art Unit: 2626 

The applicant has traversed the drawing objection of Fig. 15G for reasons similar 
to those directed to Fig. 15G in the description (Amendment, Page 53). In regards to 
such arguments, see the corresponding response directed towards Fig. 15G in the 
above specification discussion.' 

Lastly, in regards to the drawing objection directed towards Fig. 13B, the 
applicant argues that they are unable to make sense of the objection because 13B was 
submitted as a newly added drawing and cannot be different from what was originally 
shown (Amendment, page 53). In response, the examiner points out that the applicant 
admits that "FIG. 13B is also taken from originally filed FIG. 3A" (Amendment, Page 43). 
Thus, the applicant appears to understand that this Figure is identical to Fig. 3A and 
amended Fig. 3A (which was also noted as containing new matter). Thus, the items 
are different from those originally shown not with respect to a previous Fig. 13B, but 
with respect to original Fig. 3A, which the applicant expressly recognizes as being the 
figure from which Fig. 13B is derived. Therefore, Fig. 13B is directed to new matter for 
reasons similar to Claim 3A. 

Response to Arguments 

15. Applicant's arguments directed to the prior art rejections have been fully 
considered but they are not persuasive for the following reasons: 

The applicant begins by traversing the art rejection under 35 U.S.C. 102(e), 
alleging that Watanabe et al (U.S. Patent: 6,035,267) fails to teach "Determining 
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whether the voice information includes command information and if so then " because 
the performance of the proceeding steps are contingent on the trailing conditional 
statement "and if so then", which the applicant alleges was ignored by the examiner and 
not taught by Watanabe (Amendment, Page58). 

In response, the examiner notes that such a conditional statement is taught by 
Watanabe. More specifically, Watanabe teaches that a user input in a natural language 
is received by the interactive command recognition system and analyzed to determine 
whether it includes command information (Col. 4, Line 65- Col. 5, Line 16; and Col. 5, 
Lines 60-62). Looking to Fig. 2, it can be seen that the speech recognition in Watanabe 
is dependent upon various recognition knowledge databases that include various known 
system commands (such as Element 11). If a user were to speak a non-sense 
statement into the system or a command outside those known to the system, the 
system would inherently be unable to proceed with its meaning extraction steps, as it 
extracts meaning within its scope of knowledge. In this way, Watanabe does anticipate 
the conditional statement that the applicant has alleged that this reference fails to teach. 

The applicant continues to argue the aforementioned conditional statement by 
alleging that Watanabe discloses and the examiner admits that a goal processing is 
performed regardless of whether the voice information contains command information 
(Amendment, Page 59). In response, the examiner once again notes that there is 
nothing in Watanabe that states that the processing subsequent to determining that a 
voice input is a command is always performed, nor was there any admission by the 
examiner to the effect of a lacking conditional statement. Watanabe, as was previously 
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discussed, accepts a natural language speech input from a user (Fig. 2, Element 10) 
and attempts to determine if it is a command by checking for and extracting semantic 
information (Col. 5, Lines 1-16). In the case of a statement containing nonsense or a 
vocabulary outside of that known to the system, no meaning could be extracted. Thus, 
although not claimed, Watanabe does teach the negative side of this conditional 
statement. This non-claiming is important to note here. The applicant is arguing that 
Watanabe fails to teach the conditional statement "and if so then" thus implying that 
Watanabe does not teach if not then not performing the subsequent steps. Such 
processing is not claimed. It is important to note the distinction between what the 
applicant is arguing and what is actually being claimed. The limitation in question reads 
"Determining whether the voice information includes command information, and if so 
then". In the case of Watanabe, his invention does feature a step that checks for 
semantic command information (Col. 5, Lines 1-16) and if it detects this information (i.e., 
"if so then") performs the applicant's claimed subsequent steps (See Prior OA, Pages 
11-12). There is no conditional statement in the claims reading "if not then" as the 
applicant appears to be alleging and which is the point of contention here. In response 
to applicant's argument that the references fail to show certain features of applicant's 
invention, it is noted that the features upon which applicant relies (i.e., the "if not then" 
statement) are not recited in the rejected claim(s). Although the claims are interpreted 
in light of the specification, limitations from the specification are not read into the claims. 
See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Thus, for this 
additional reason this argument has been fully considered, but is not convincing. 
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In regards to the same "Determining" limitation, the applicant argues that this 
limitation is not even considered by Watanabe because various processes associated 
with determining presented in the specification are allegedly not taught by Watanabe 
which supposedly only teaches command processes (Amendment, Page 60). In 
response, the examiner notes that the applicant appears to be interpreting their claim 
scope much more narrowly than what is actually quite broadly claimed. The limitation in 
question simply states "Determining whether the voice command includes command 
information". In Watanabe an input statement of a user is analyzed syntactically and 
semantically to determine if command information (terms, structures, etc.) is present 
(Col. 5, Lines 1-16; and Col. 5, Lines 60-67). The applicant even admits that Watanabe 
provides command determination (Amendment, Page 60). Thus, this argument has 
been fully considered, but is not convincing. 

The applicant subsequently focuses on the limitation regarding "determining one 
or more specificities corresponding to the command information" and argues that 
Watanabe fails to teach this recitation (Amendment, Page 60). Specifically, the 
applicant first argues that Watanabe does not even "remotely relate to anything more 
than matching the so-called "user goal" with a corresponding predetermined system 
operation" (Amendment, Page 61). In response, the examiner notes that it is this 
teaching that anticipates the applicant's recited specificities. Watanabe teaches that 
upon extracting a user based objective (Col. 5, Lines 1-16 and Col. 5, Line 60- Col. 6, 
Line 3), voice information is further analyzed with respect to a specific system function 
or access/functional component (Col. 5, Lines 1-16; Col. 6, Lines 13-21). Since specific 
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interactive system functions or "specificities" to which the voice input signal is 
interpreted as being directed to is determined by Watanabe, this argument has been 
fully considered, but is not convincing. The applicant also makes a traversal with 
respect to Watanabe's user goal allegedly corresponding to the system goal and makes 
the conclusion that Watanabe's correspondence here contradicts the claimed specificity 
determination. In response, the examiner notes that, as was pointed out above, 
Watanabe specifically analyzes a voice input to extract a specific system meaning from 
a user's naturally phrased optionally structured input. It is unclear, how any 
correspondence between user and system goals would contradict anything in the claims 
(for example, there is no recitation of a disagreement between the user based 
objectives and the specificities in the claimed invention). Thus, this argument has been 
fully considered, but is not convincing. Finally, in regards to this limitation, the applicant 
argues that nowhere in Watanabe are modifiable "specifiable commands "discussed 
(Amendment, Pages 61-62). Although the applicant is arguing features which are not 
claimed, the examiner notes that a varying degree of user specificity is nonetheless 
taught by Watanabe. In Watanabe, a user is capable of inputting any type of naturally 
spoken statement in order to accomplish a goal (Col. 5, Lines 1-16). Through system 
goal determining meaning of the user's naturally varying speech input is extracted an a 
specific meaning rendered (Col. 6, Lines 13-21). While the system of Watanabe has a 
general scope of extraction, it is important to note that his system does not operate on a 
restrictive grammar as is alleged by the applicant because it allows meaning extraction 
from a natural language input. Thus, this argument has been fully considered, but is not 
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convincing based on the fact that the applicant is arguing features which are not 
claimed and that Watanabe actually teaches these types of commands. 

The applicant next makes a statement that the Watanabe reference cannot be 
modified because if it was it would "clearly" be inoperable and not enabled 
(Amendment, Page 62). While such a statement is moot because the maintained art 
rejection is directed to one under anticipation via 35 U.S.C. 102(e), this argument is 
baseless. A broad statement such as this one made by the applicant cannot be made 
without first considering a set of possible modifications. 

The applicant continues their argument by turning to the limitation regarding 
"determining a conversant command execution corresponding to use-based objectives 
and the specificities" (Amendment, Pages 62-64), which they allege is not taught by 
Watanabe. Specifically, the applicant cites the section of Watanabe applied in the art 
rejection and argues that Watanabe merely discloses that a "command execution is 
conducted using a sequence of first "feasible" actions" and fails to discuss the 
command execution being a "conversant one" (Amendment, Page 63). The applicant 
also attempts to argue that the examiner has not even admitted that the execution in 
Watanabe is conversant (Amendment, Pages 63-64). 

In response, the examiner notes that Watanabe analyzes an input corresponding 
to "natural language speech" (Col. 5, Line 61). In the speech analysis, Watanabe's 
system extracts user and system goals (Col. 5, Lines 1-16) and then proceeds to 
determine an action execution based upon these goals (Col. 5, Lines 29-50; and Col. 6, 
Line 36- Col. 7, Line 5). It is important to note that the executed actions correspond to 
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the user's input "natural language speech". In this manner, the actions performed by 
Watanabe are "conversant". Also, the examiner points out that the brief description 
provided in the cited sections of Watanabe is directly mapped to the "conversant 
command execution" so it was clear that the examiner did recite that conversant 
commands were taught by Watanabe. Thus, these arguments have been fully 
considered, but are not convincing. In response to the applicant's commentary 
regarding the modification of Watanabe (Amendment, Page 64), the examiner notes 
that this argument has already been addressed. As such, see the corresponding 
associated response. 

Thus, based on at least the aforementioned reasons, the applicant's arguments 
directed towards claim 1 have been fully considered, but are not convincing. 

The applicant traverses the art rejections directed towards claims 2-3 for reasons 
similar to claim 1 (Amendment, Page 65). In regards to such arguments, see the 
preceding response directed towards claim 1 . 

The applicant traverses the art rejections directed towards dependent claims 5-8, 
12, 15-36, 38-39, and 42-44 for reasons similar to claim 1 (Amendment, Page 65). In 
regards to such arguments, see the preceding response directed towards claim 1 . 

The applicant traverses the art rejections directed towards claims 4, 9-10, and 
13-14 for reasons similar to claim 1 (Amendment, Pages 66-67). In regards to such 
arguments, see the preceding response directed towards claim 1 . 
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The applicant traverses the art rejection directed towards dependent claim 40 for 
reasons similar to claim 1 (Amendment, Pages 67-68). In regards to such arguments, 
see the preceding response directed towards claim 1 . 

Thus, for at least the preceding reasons, the applicant's arguments have been 
fully considered, but are not convincing. 

Specification 

16. The disclosure is objected to because of the following informalities 

On page 35, "use objective" should be changed to -user objective-- in order to 
make this terminology more consistent with that utilized in the reminder of the 
specification and because the sentence in which it is contained specifies a user-based 
objective. 

Appropriate correction is required. 

1 7. The amendment filed 4/3/2008 is objected to under 35 U.S.C. 1 32(a) because it 
introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment 
shall introduce new matter into the disclosure of the invention. The added material 
which is not supported by the original disclosure is as follows: 

The applicant's amendment to the specification beginning at page 8, Line 3 
includes new matter because it refers back to Figs. 3A, 13B, 15F, and 15 G, which 
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contains new matter. The specific reasons as to why these drawings are directed to 
new matter are described in further detail below. 

The applicant's amendment to the specification beginning at page 58, line 5 
includes new matter because it refers back to Figs. 3A and 13B, which contains new 
matter. Figs. 3A and 13B contains elements that were not disclosed in the specification 
as filed (for example, entertainment systems). 

This amendment also references other Figures containing new matter- 15F (no 
mention of use of image recognition type processing for user identification), and 15G 
(no mention of trapping database or selective output portion). The descriptions of these 
figures are found in the amended specification beginning at Page 8, Line 3. 

The applicant's amendment to the specification beginning at page 22, line 14 
includes new matter. This portion of the specification corresponds to originally 
submitted Figure 3a. The specification as originally filed did not make any mention of 
head/body position recognition or gaze to affect machine portions. Although the 
specification does consider the entry of non-speech gestures, the specification makes 
no mention of what is included in these gestures or how they can be recognized and 
processed. 
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The applicant's amendment to the specification beginning at page 27, line 4 
includes new matter because it refers back to Fig. 15G, which contains new matter. 
The specification as filed makes no mention of the interpreter processor connected to a 
"trapped" database and also having a selective output portion (i.e., no output or 
processed output). 

The applicant's amendment to the specification beginning at page 30, line 1 1 
incorporates new matter. The specification as originally filed is silent on the newly 
recited types of lesser related intermittent tasks because there is no mention of 
controlling lights, television, stereo/gaming systems, etc. 

The applicant's amendment to the specification beginning at page 32, line 29 
includes new matter because it refers back to Fig. 13B, which contains new matter. Fig. 
13B contains elements that were not disclosed in the specification as filed (for example, 
entertainment and gaming systems). 

The applicant's amendment to the specification beginning at page 47, line 10 
includes new matter because it redefines the context and task engines by deleting part 
of their definition from the specification. 

The applicant's amendment to the specification beginning at page 58, line 5 
includes new matter because it refers back to Figs. 3A and 13B, which contains new 
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matter. Figs. 3A and 13B contain elements that were not disclosed in the specification 
as filed (for example, entertainment systems). 

The applicant's amendment to the specification beginning at page 65, line 22 
includes new matter. More specifically, the specification as originally filed did include a 
command input resolver in the drawings, but is completely silent as to how it would 
function. The added description of this element was thus not mentioned and is directed 
to new matter. Similar issues apply to the added enhancement engine (616) 
components (i.e., machine switcher, query engine, choice/error feedback). 

Applicant is required to cancel the new matter in the reply to this Office Action. 

18. The amendment filed 5/5/2009 is objected to under 35 U.S.C. 1 32(a) because it 
introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment 
shall introduce new matter into the disclosure of the invention. The added material 
which is not supported by the original disclosure is as follows: 

In the amendment beginning at Page 74, Line 3, regarding the added paragraph 
describing Fig. 10b, the "smart phone" was not shown in original Figure 10B (which was 
initially unnumbered and found on the drawing page beginning the "Control Interface") 
nor was there any discussion of a "smart phone" in the originally filed specification. 
Looking to original Fig. 10B, a black object can be seen next to a user's ear which 
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appears to be a phone. New Fig. 10B which was filed on 4/3/2008 provided a better 
representation of the originally submitted hand drawing, but there was nothing in the 
original drawing or the specification that indicated that the phone was a "smart phone". 
Thus, the inclusion of "smart phone" directs this paragraph towards new matter. 

In the amendment beginning at Page 74, Line 3, regarding the last sentence of 
the added paragraph describing Fig. 10C, the examiner notes that while this Figure was 
shown in the originally filed specification as being Fig. 5, there was nothing in the figure 
or the specification to indicate that the process could be reversed. In fact, original Fig. 5 
explicitly indicated uni-directions left-to-right operations. Looking at original Fig. 5, an 
arrow labeled "conversational commands" can be seen pointing to the right, while the 
output of the converter can also been seen as pointing to the right and is labeled "To 
Voice Command Interpreter". It is clear from looking at this drawing then, that there was 
no reverse process even contemplated in the originally filed specification/drawings. As 
the paragraph describing 10C includes reverse processing, it is directed towards new 
matter. 

In the amendment beginning at Page 74, Line 3, regarding the added paragraph 
describing Fig15G. (a figure which was noted was being directed to new matter), the 
applicant has compounded the new matter issue by adding a description of this 
drawing. As this drawing is directed towards new matter, these added descriptions are 
likewise objected to as being directed towards new matter. 
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Applicant is required to cancel the new matter in the reply to this Office Action. 



Drawings 



1 9. The drawings are objected to because Figs. 3A (different items not originally 
shown), 15G (trapped database, selective output portion), and 13B (different items not 
originally shown). 

Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are required in reply to 
the Office action to avoid abandonment of the application. Any amended replacement 
drawing sheet should include all of the figures appearing on the immediate prior version 
of the sheet, even if only one figure is being amended. The figure or figure number of an 
amended drawing should not be labeled as "amended." If a drawing figure is to be 
canceled, the appropriate figure must be removed from the replacement sheet, and 
where necessary, the remaining figures must be renumbered and appropriate changes 
made to the brief description of the several views of the drawings for consistency. 
Additional replacement sheets may be necessary to show the renumbering of the 
remaining figures. Each drawing sheet submitted after the filing date of an application 
must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" 
pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 
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Claim Objections 

20. Claims 1-44 are objected to because of the following informalities: 

Claims 1, 3, and 4 recite "use-based" objectives, but it appears this limitation 
should state -user based objectives-- (for example, see specification, Page 10). 

The remainder of the dependent claims fail to overcome the preceding objections 
and thus, are similarly objected to by virtue of their dependency. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 101 

21. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

22. Claims 1-3, 5-10, 12-36, and 38-44 are rejected under 35 U.S.C. 101 because 
the claimed invention is directed to non-statutory subject matter. 

Claim(s) 1 and associated dependent claims 2, 5-10, 12-36, and 38-44 is/are 
rejected under 35 USC 101 as not falling within one of the four statutory categories of 
invention. While the claim(s) recite a series of steps or acts to be performed, a statutory 
"process" under 35 USC 101 must (1) be tied to another statutory category (such as a 
manufacture or a machine), or (2) transform underlying subject matter (such as an 
article or material) to a different state or thing. The instant claim(s) neither transform 
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underlying subject matter (i.e., the claim fails to transform physical matter to a different 
state or thing) nor positively recite structure associated with another statutory category 
(i.e., the claim does not explicitly require the use of any type of physical hardware and 
could be performed by a human. For example- a human could receive a request for 
assistance from another machine user, understand their voice request, determine the 
goal of the user based on the understanding, determine the object of the goal based on 
the understanding, and perform an understood function to assist the user.), and 
therefore do not define a statutory process. Also claim 2 is appears to be directed to a 
system and method for forming this system. A method and system does not fall within 
one of the four statutory categories of invention (i.e., process, machine, manufacture, or 
composition of matter) because it claims two categories. "Such claims may also be 
rejected under 35 U.S.C. 101 based on the theory that the claim is directed to neither a 
"process" nor a "machine," but rather embraces or overlaps two different statutory 
classes of invention set forth in 35 U.S.C. 101 which is drafted so as to set forth the 
statutory classes of invention in the alternative only. Id. at 1551 ." See MPEP 
2173.05(p) (II). 

Although claim(s) 3 appears to fall within a statutory category (i.e., apparatus), 
claim(s) 3 encompasses nothing more than logic/software modules as per the 
specification ("System 200 element implementations can include.... software", Page 19). 
Thus, claim(s) 3 are directed to non-statutory subject matter because their scope 
includes a computer program embodiment, an abstract data structure which does not 
fall within one of the four statutory categories (i.e., it is directed to a program perse). 
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See also MPEP § 2106.IV.B.1 .a. Data structures not claimed as embodied in computer 
readable media are descriptive material perse and are not statutory because they are 
not capable of causing functional change in the computer. See, e.g., Warmerdam, 33 
F.3d at 1361, 31 USPQ2d at 1760 (claim to a data structure perse held nonstatutory). 
Such claimed data structures do not define any structural and functional 
interrelationships between the data structure and other claimed aspects of the invention, 
which permit the data structure's functionality to be realized. In contrast, a claimed 
computer readable medium encoded with a data structure defines structural and 
functional interrelationships between the data structure and the computer software and 
hardware components which permit the data structure's functionality to be realized, and 
is thus statutory. Similarly, computer programs claimed as computer listings perse, i.e., 
the descriptions or expressions of the programs are not physical "things." They are 
neither computer components nor statutory processes, as they are not "acts" being 
performed. Such claimed computer programs do not define any structural and 
functional interrelationships between the computer program and other claimed elements 
of a computer, which permit the computer program's functionality to be realized. 



Claim Rejections - 35 USC §112 



23. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 



The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 
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24. Claims 2 and 25-26 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

In claim 2, it is uncertain/unclear how the method of claim 1 produces an 
interfacing system. Claim 1 is directed towards a claimed interfacing method and it is 
an uncertain how a method for determining user commands forms/creates a system as 
claim 2 recites. A system requires structural elements, which claim 2 lacks. Thus, 
claim 2 is indefinite. Also, in the case that the applicant is intending to claim a system 
and its use- A single claim which claims both an apparatus and the method steps of 
using the apparatus is indefinite under 35 U.S.C. 112, second paragraph. IPXL Holdings 
v. Amazon.com, Inc., 430 F.2d 1377, 1384, 77 USPQ2d 1140, 1145 (Fed. Cir. 2005); 
Ex parte Lyell, 17 USPQ2d 1548 (Bd. Pat. App. & Inter. 1990) (claim directed to an 
automatic transmission workstand and the method of using it held ambiguous and 
properly rejected under 35 U.S.C. 112, second paragraph). See MPEP 2173.05(p) (II). 

In claim 25, it is uncertain whether the term in parenthesis is actually a part of 
the claim or is alternative processing conducting means. The claim should be amended 
accordingly to better indicate what is meant by the parenthetical statement (for example 
-an expression usage-). Claim 26 fails to overcome this 35 U.S.C. 112, second 
paragraph issue, and thus, is also rejected as being indefinite. 

25. The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
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art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

26. Claim 28 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. The specification as originally filed makes no 
mention of trapping processing as is recited in claim 28. 

Claim Rejections - 35 USC § 102 

27. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1 ), (2), and (4) of section 371 (c) of this 
title before the invention thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act 
of 1999 (AIPA) and the Intellectual Property and High Technology Technical 
Amendments Act of 2002 do not apply when the reference is a U.S. patent resulting 
directly or indirectly from an international application filed before November 29, 2000. 
Therefore, the prior art date of the reference is determined under 35 U.S.C. 102(e) prior 
to the amendment by the AIPA (pre-AlPA 35 U.S.C. 102(e)). 
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28. Claims 1-3, 5-8, 12, 15-36, 38-39, and 42-44 are rejected under 35 
U.S.C. 102(e) as being anticipated by Watanabe et al (U.S. Patent: 6,035,267). 
With respect to Claim 1, Watanabe discloses: 

Receiving voice information corresponding to at least one machine user 
(received input speech in a natural language from a user, Col. 4, Line 65- Col. 5, Line 
16; Col. 5, Lines 60-62); and 

Processing the voice information, the processing including: 

Determining whether the voice information includes command information 
(identifying semantic terms related to system commands, Col. 5, Lines 1-16); 

Determining one or more use-based objectives corresponding to the voice 
information (a user goal is determined for the voice input, Col. 5, Lines 1-16; and Col. 5, 
Line 60- Col. 6, Line 3); 

Determining one or more specificities corresponding to the command information 
(a system goal is determined, which corresponds to the user's command, Col. 5, Lines 
1-16; Col. 6, Lines 13-21); and 

Determining a conversant command execution corresponding to the use-based 
objectives and the specificities (determining an action execution based on the goals, 
Col. 5, Lines 29-50; and Col. 6, Line 36- Col. 7, Line 5). 

With respect to Claim 2, Watanabe further discloses: 

An interfacing system formed according to the method of claim 1 (processing 
apparatus, Col. 4, Lines 65-66). 
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Claim 3 contains subject matter similar in scope to claims 1-2, and thus, is 
rejected under similar rationale. 

With respect to Claim 5, Watanabe further discloses: 

The voice information comprises at least one of: a conversant voice command 
recitation of one or more users, a non-conversant voice command recitation of one or 
more user and monitored speech of one or more users (natural language speech 
commands, Col. 5, Lines 60-62). 

With respect to Claim 6, Watanabe further discloses: 

The determining one or more use-based objectives comprises determining that at 
least one voice information portion of the voice information corresponds to at least one 
of a user task and a user goal (user goal identification from input speech information, 
Col. 5, Lines 1-16; and Col. 5, Line 60- Col. 6, Line 3). 

With respect to Claim 7, Watanabe further discloses: 

The determining one or more use-based objectives comprises determining that at 
least a voice information portion of the voice information corresponds to at least one of 
a group task and a group goal (variations of multiple user speech commands are 
grouped under a goal, Col. 5, Lines 51-59). 

With respect to Claim 8, Watanabe further discloses: 

The determining one or more specificities includes determining one or more 
explicit specificities as corresponding to one or more explicitly recited voice information 
portions of the voice information (system goals directly correspond to the user speech 
input, Col. 5, Line 54- Col. 6, Line 21). 
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With respect to Claim 12, Watanabe further discloses: 

Determining one or more machine portions of the one or more hosted machines 
for effecting the one or more use-based objectives (determining system portions for 
effecting user goals, Col. 7, Lines 6-25). 

With respect to Claim 15, Watanabe further discloses: 

At least a portion of the processing is conducted in accordance with one or more 
of a context, a conversant context, an interaction, an approach, and a scenario (context 
is utilized in conversational processing, Col. 6, Lines 54- Col. 7, Line 5). 

With respect to Claim 16, Watanabe further discloses: 

Determining at least one likely further use-based objective that may be 
determined in accordance with further received voice information (further request for 
information to which a user responds with additional voice information, Col. 6, Line 54- 
Col. 7, Line 5; and user goal determination, Col. 5, Lines 1-16); and 

Conducting at least a portion of the processing in accordance with the at least 
one likely further use-based objective (processing is continually performed according to 
user goals, Col. 7, Lines 52-62). 

With respect to Claim 17, Watanabe further discloses: 

Processing is conducted in accordance with at least one of a processing history, 
a user habit, and a user tendency of at least one user (history information is utilized in 
processing, Col. 6, Lines 54-67). 

With respect to Claim 18, Watanabe further discloses: 
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AT least a portion of the processing is conducted in accordance with a content 
characterization, the content characterization including at least one of an information 
type, an information use, an information application, and an information purpose 
(information application including tagged information for searching, Col. 7, Lines 6-25). 

With respect to Claim 19, Watanabe further discloses: 

The information type is selected from a group including commands, data, 
biometric data, dictation, and specific data type (specific value associated with a type of 
data, Col. 7, Lines 6-25). 

With respect to Claim 20, Watanabe further discloses: 

The information type is selected from a group including silence, misstatement, 
mis-recitation, private information, and confidential information (processing involving 
classification of a misstatement or mis-recitation in the form of an illegal input, Col. 7, 
Lines 6-25). 

With respect to Claim 21, Watanabe further discloses the processing of business 
information (hotel, airline, etc, Col. 7, Lines 6-25). 

With respect to Claim 22, Watanabe further recites: 

The information application is selected from a group including charting, home 
control, calendaring, vehicle operation, communication, multimedia production, media 
presentation, and document production (communication and scheduling flights or hotel 
stays, Col. 7, Lines 6-25). 

With respect to Claim 23, Watanabe further recites: 
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The information purpose is selected form a group including a particularized 
objective and a subject matter of a user to which at least one of a command portion, a 
data portion, a dictation portion is determined to be directed (command directed to a 
user/system goal, Col. 5, Line 60- Col. 6, Line 21; and Col. 7, Lines 52-62). 

With respect to Claim 24, Watanabe further discloses filling-in or reviewing 
booking form information (Col. 7, Lines 6-25). 

With respect to Claim 25, Watanabe further discloses: 

The processing is conducted in accordance with a usage (expression 
characterization) corresponding to at least a portion of the voice information (processing 
carried out according to expression characterization via language syntax, Col. 5, Line 
60- Col. 6, Line 3). 

With respect to Claim 26, Watanabe further discloses that the manner of 
expression is selected from a group including language, dialect and colloquialism, 
inflection, biometrics, physical gesture, and non-speech expression (expression in a 
user's natural language, Col. 5, Line 60- Col. 6, Line 3). 

With respect to Claim 27, Watanabe further discloses disabling illegal speech 
input portions and enabling the acceptable portions (Col. 7, Lines 6-25). 

With respect to Claim 28, Watanabe further discloses trapping and muting and 
substitution of incorrect/insufficient speech inputs (Col. 6, Line 54- Col. 7, Line 25). 

With respect to Claim 29, Watanabe further recites local speech vocabulary word 
determination (i.e., speech recognition) involving syntax understanding (Col. 5, Line 60- 
Col. 6, Line 3). 
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With respect to Claim 30, Watanabe further discloses: 

The processing further comprises determining at least one of an operational 

mistake, corrective action, and implicit user assisting in accordance with at least one of 

a content characterization and an expression characterization (user assistance provided 

to a user in response to input speech characterization, Col. 6, Line 54- Col. 7, Line 5). 

With respect to Claim 31, Watanabe further discloses: 

Determining that a portion of the voice information corresponds with an anti-alias 
(determining a portion of a user's voice input that is a name that is not an alias, Col. 5, 
Line 51- Col. 6, Line 3), the anti-alias comprising an anti-alias designation indicating at 
least one specific target of the anti-alias (anti-alias name designation associated with a 
target under a user goal, Col. 5, Line 51- Col. 6, Line 3; and Figs. 3-4); and 

Resolving the anti-alias (resolving the meaning of the non-alias using natural 
language understanding, Col. 5, Line 51- Col. 6, Line 3). 

With respect to Claim 32, Watanabe further discloses: 

The resolving the anti-alias comprises determining at least one specific target in 
accordance with at least one of a current class membership, a current title, and a 
currently performed function (determining the anti-alias target based on a current user 
goal function context, Col. 5, Line 51- Col. 6, Line 3; and Col. 7, Lines 26-62); 

The at least one specific target and the at least one of a class, title, and function 
correspond with one or more of explicit and implicit specificities, the one or more 
specificities further corresponding to at least a portion of the voice information and the 
voice information corresponds with one or more recitations (explicit target is associated 
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with the alias under a user goal and further system goal and is derived from user 
speech, Col. 5, Line 51- Col. 6, Line 21; and Col. 7, Lines 26-62). 

With respect to Claim 33, Watanabe further discloses: 

The anti-alias designation indicates at least one of a target classification 
designation and a source of target resolution information in accordance with which the 
anti-alias may be resolved (the target of the alias is associated with a term having a field 
class designation, Col. 5, Line 51- Col. 6, Line 3); and 

The anti-alias further comprises: an association of the target-anti-alias 
designation with at least one specificity (anti-alias is associated with a system goal 
information, Col. 5, Line 60, Col. 6, Line 21). 

With respect to Claim 34, Watanabe further discloses: 

The anti-alias designation indicates possession (the designation points back to 
the alias, indicating possession of that term under a user goal, Col. 5, Line 51- Col. 6, 
Line 3; and Fig. 4). 

With respect to Claim 35, Watanabe further discloses: 

The determining a conversant command execution includes designating at least 
one machine portion for executing at least a portion of the voice information 
(designating a machine portion upon which the action is to be performed, Col. 7, Lines 
6-25); 

The designating is conducted in accordance with at least one of a not explicitly 
stated ("implied") specificity and the use-based objective (designating a machine portion 
through the user goal, Col. 6, Line 54- Col. 7, Line 25), thereby enabling one or more of 
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feedback corresponding to the transition or non-transition, completion of a designation 
objective and preparation corresponding to a likely successive user recitation 
(communication feedback regarding completion of a goal or transition/non-transition to 
another system state, Col. 7, Lines 1-62); and 
Wherein: 

the designating causes a designation of a current machine portion to transition to 
a non-current machine portion, causing the executing to be conducted by invoking 
operabilities of the non-current machine portion (switching to another system processing 
portion through a changed goal, Col. 6, Line 54- Col. 7, Line 5; and Col. 7, Lines 52-62). 
With respect to Claim 36, Watanabe further discloses: 
The designating is further conducted in accordance with at least one of an 
operational history, a user habit and a user tendency (employment of an interaction 
history, Col. 6, Line 54- Col. 7, Line 5). 

With respect to Claim 38, Watanabe further discloses an application program 
having a GUI (Col. 7, Lines 6-25). 

With respect to Claim 39, Watanabe further discloses: 
The voice information includes previously received voice information and 
currently received voice information (current speech and speech corresponding to a 
previous iteration stored in a history, Col. 5, Line 60- Col. 6, Line 3; and Col. 6, Line 54- 
Col. 7, Line 5); 

At least one of the determining a use-based objective, the determining one or 
more specificities and the determining a conversant execution includes determining that 
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at least a portion of the currently received voice information corresponds with at least a 
portion of the previously received voice information ('determining a correspondence") 
(the processing involves looking at a history storage to determine arguments that exits 
from a previous speech section, Col. 6, Line 54- Col. 7, Line 5); 

At least one of the determining a use-based objective, the determining one or 
more specificities and the determining a conversant execution includes processing at 
least a portion of the previously received voice information and the currently received 
voice information as a discontinuous recitation of a use-based objective (associating 
mandatory arguments from multiple speech iterations as a discontinuous recitation of a 
user goal, Col. 6, Line 54- Col. 7, Line 5; and Col. 7, Lines 26-62). 

With respect to Claim 42, Watanabe further discloses discontinuous argument 
items of a goal group that can be processed in different iterations (Col. 6, Line 54- Col. 
7, Line 5). 

With respect to Claim 43, Watanabe further discloses a missing element that 
cues an action is determined to be executed (Col. 6, Line 54- Col. 7, Line 46). 

With respect to Claim 44, Watanabe teaches that when a missing argument is 
input and a user statement is complete, the information in the history storage is acted 
upon to execute an appropriate command (Col. 6, Line 54- Col. 7, Line 25). 
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Claim Rejections - 35 USC § 103 

29. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

30. Claims 4, 9-10, and 13-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Watanabe et al in view of Coffman et al (U.S. Patent: 6,377,913). 

With respect to Claim 4, Watanabe discloses the user-goal based speech 
processing method, as applied to Claim 1 . Watanabe does not explicitly teach method 
implementation as a program stored on a computer readable medium, however, 
Coffman recites implementing a speech recognition method as software stored in 
memory (Col. 3, Lines 30-34). 

Watanabe and Coffman are analogous art because they are from a similar field 
of endeavor in speech controlled systems. Thus, it would have been obvious to a 
/person of ordinary skill in the art, at the time of invention, to modify the teachings of 
Watanabe with the concept of method implementation as a computer program as taught 
by Coffman in order to allow the speech processing system of Watanabe to be 
implemented in any general purpose computer (Coffman, Col. 3, Lines 32-35). 

With respect to Claim 9, Watanabe discloses the user-goal based speech 
processing method, as applied to Claim 1 . Watanabe does not explicitly teach the 
execution of implied specificities, however, Coffman teaches the ability to execute a 
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command with implied user information that is not actually included in the spoken input 
(Col. 4, Line 35- Col. 5, Line 32) to better understand a user and customize system 
responses (Coffman, Col. 6, Lines 54-60). 

With respect to Claim 10, Coffman further discloses processing performed 
according to user identification (Col. 4, Line 35- Col. 5, Line 11). 

With respect to Claim 13, Coffman further discloses processing conducted in 
accordance with a machine association with a user (Col. 4, Line 35- Col. 5, Line 11). 

With respect to Claim 14, Coffman further discloses processing conducted in 
accordance with a machine portion use (Col. 4, Lines 35- Col. 5, Line 52). 

31 . Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Watanabe et al in view of Coffman et al (U.S. Patent: 6,799,169). 

With respect to Claim 40, Watanabe discloses the speech processing system 
that relates past and current commands, as applied to claim 39. Also, Watanabe further 
discloses: 

The determining a correspondence determines that the currently received 
information at least partially completes the previously received information by including 
one or more corresponding specificities having a same input type characterization 
(arguments from past and present required to complete a system goal, Col. 6, Line 54- 
Col. 7, Line 5); 

The determining a correspondence determines that the currently received 
information at least partially completes the previously received information by providing 
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one or more corresponding specificities having a different input type characterization 
(arguments in goal frame filling portions that have a different input syntax 
characterization, Col. 5, Line 60- Col. 6, Line 3; and Col. 6, Line 54- Col. 7, Line 5); and 
The determining a correspondence determines that at least one of the previously 
recited information and the currently received information comprises a partial recitation 
(portions of a complete state correspond to each voice portion, Col. 6, Line 54- Col. 7, 
Line 5). 

Watanabe does not specifically teach determine a correspondence according to 
a linking indicator, however, Coffman '169 recites: 

The determining a correspondence determines that the previously received 
information and the currently received voice information correspond with an interrupted 
single recitation (determining an interrupted speech recognition session, Col. 5, Lines 1- 
20); and 

The determining a correspondence comprises determining that at least one of 
the previously received voice information and the currently received voice information 
includes a linking indicator (response type indicator associated with the past and 
present voice information, Col. 4, Lines 28-54). 

Watanabe and Coffman '169 are analogous art because they are from a similar 
field of endeavor in speech controlled systems. Thus, it would have been obvious to a 
/person of ordinary skill in the art, at the time of invention, to modify the teachings of 
Watanabe with the association means taught by Coffman '169 in order to allow the user 
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the option to complete any action at a later time or not at all (Coffman '169, Col. 5, Lines 
18-21). 

32. Claim 41 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Watanabe et al in view of Goldhor et al (U.S. Patent: 5,970,448). 

With respect to Claim 41, Watanabe discloses the speech processing system 
that relates past and current commands, as applied to claim 39. Watanabe does not 
specifically suggest the processing of claim 41 involving a linking indicator, however, 
Goldhor discloses: 

Conducting independent processing corresponding to at least a portion of the 
previously received information (performing speech recognition processing on a portion 
of the previous voice information, Col. 8, Lines 24-39); 

Preserving the independent processing if the currently received voice information 
includes a linking indicator (a next speech entry is provided having a connecting pointer 
and a previous speech input processing is maintained, Col. 8, Lines 40-56; and Col. 6, 
Lines 6-15); and 

Modifying at least a portion of a result obtained in accordance with the 
independent processing if the currently received voice information does not include a 
linking indicator (command is received that modifies the previous processing result, Col. 
9, Lines 15-35). 

Watanabe and Goldhor are analogous art because they are from a similar field of 
endeavor in speech controlled systems. Thus, it would have been obvious to a /person 
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of ordinary skill in the art, at the time of invention, to modify the teachings of Watanabe 
with the pointer-related processing taught by Goldhor in order to allow a user to 
continue an interactive speech session at a later time (Goldhor, Col. 5, Lines 48-56). 

Conclusion 

33. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: See PTO-892. 

34. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 

272- 7632. The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off 
Alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached at (571) 272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

/James S. Wozniak/ 

Primary Examiner, Art Unit 2626 



